Papers with Dense embeddings
Interpret and Control Dense Retrieval with Sparse Latent Features (2025.naacl-short)
Copied to clipboard
| Challenge: | Dense embeddings deliver strong retrieval performance but lack interpretability and controllability. |
| Approach: | They propose a novel approach using sparse autoencoders to interpret and control dense embeddings via latent sparsity. |
| Outcome: | The proposed approach retains the same retrieval accuracy as the original dense vectors, affirming their faithfulness. |
Beyond instruction-conditioning, MoTE: Mixture of Task Experts for Multi-task Embedding Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Instruction-conditioning has become the dominant approach for embedding specialization, but its direct application to low-capacity models imposes representational constraints that limit the performance gains derived from specialization. |
| Approach: | They propose a mixture of task experts transformer block which leverages task-specialized parameters trained with Task-Aware Contrastive Learning to enhance the model’s ability to generate specialized embeddings. |
| Outcome: | The proposed model achieves 64% higher performance gains in retrieval datasets (+3.27 +5.21) and 43% higher performance gain across all datasets (+1.81 2.60). |